-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add queueing and multi process support #48
Conversation
Peddle
commented
Dec 7, 2023
•
edited
Loading
edited
- Adds ability to queue multiple requests for better local dev experience
- adds request id to all logs
- adds ability to configure number of processes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚗 drive by review 🚗💨
nice work, left comments 🍌
main thing is I think a lot of code can be extracted into private methods to make everything a lot cleaner to read & easier to maintain.
Doing some QA. I am just observing behavior, not saying we should fix anything just yet. |
Going to try with 100k processes, wish me luck |
Got up to 32 workers (BERT) running locally, laptop struggled but seemingly no bugs. I'd suggest
|
Tentative LGTM from me given above is addressed |